Calibration method and measurement tool

ABSTRACT

A calibration method calibrates a stereo camera. The calibration method includes: measuring a relative position between the stereo camera and an object that is placed so as to fall within an image capturing area of the stereo camera; acquiring a captured image that is captured by the stereo camera and includes the object; and determining a calibration parameter for calibrating the stereo camera based on the relative position and the captured image.

This application is a continuation application of U.S. application Ser.No. 15/114,007, filed Jul. 25, 2016, which is a National Stageapplication of PCT/JP2015/053024, filed Jan. 28, 2015, and claimspriority to Japanese Priority Application No. 2014-240206, filed Nov.27, 2014, and Japanese Priority Application No. 2014-112737, filed May30, 2014, and Japanese Priority Application No. 2014-013520, filed Jan.28, 2014. The entire contents of the above-identified applications areincorporated herein by reference.

TECHNICAL FIELD

The present invention relates to a calibration method and a measurementtool.

BACKGROUND ART

Stereo cameras capable of measuring the distance to an object are used.For example, a technology has been put into practical use to control acar by measuring the distance to an object in front of the car by usinga stereo camera (hereafter, referred to as a “car-mounted stereocamera”) that is mounted on a car. For example, the distance that ismeasured by a car-mounted stereo camera is used for giving an alarm to adriver or controlling a brake, steering, and/or the like, for thepurpose of prevention of car collision, a control on a distance betweencars, and/or the like.

Generally, car-mounted stereo cameras are often installed inside thefront windshield of cars. This is because, if a car-mounted stereocamera is installed outside the car, it needs to have a higherdurability in terms of waterproof, dust prevention, and the like. If astereo camera is installed inside the car, it captures an image of thelandscape outside the car through the front windshield. Generally, thefront windshield has a complicated curved shape, and it has a distortedshape compared to optical components such as a lens inside the camera.Therefore, the front windshield causes the captured image that iscaptured through the front windshield to be distorted. Furthermore,depending on the installation position and the installation directionwhen installing a stereo camera in the car, the characteristics of adistortion of captured images are changed. To calibrate such adistortion that is included in the captured image, it is necessary tocalculate a calibration parameter for calibrating (correcting) adistortion of the captured image by installing a stereo camera in apredetermined position of the vehicle and then using the captured imagethat is captured through the front windshield in that state.

A generally known method for calculating a calibration parameter forcalibrating a distortion of a captured image is a method that uses achart in which a specific mark, or the like, for measuring a distance isdescribed. With this method, a calibration parameter for calibrating adistortion of the captured image is calculated on the basis of thedifference between the position of a mark (object) that is on thecaptured image and that is calculated in theory on the basis of therelative position between the mark and the stereo camera and theposition of the mark on the captured image that is obtained when animage of the mark is actually captured by the stereo camera. That is, acalibration parameter is calculated, which determines a conversion so asto eliminate the difference.

Patent Literature 1 discloses a device that converts each of a pair ofimage data sets output from a pair of cameras that is included in astereo camera by using a calibration parameter that is based on thedifference in the coordinates between one set of the image data and theother set of the image data so as to adjust an optical distortion and apositional deviation of the stereo camera by image processing.

However, if there is an error in the relative position between thestereo camera and the chart, an error occurs in the coordinates of theobject that is in the captured image and that is calculated in theory;therefore, an error also occurs in a calibration parameter forcalibrating a distortion of the captured image. Particularly, there is aproblem in that an error easily occurs in the relative position betweenthe chart and the stereo camera that is installed in an object, such asa vehicle.

In consideration of the foregoing, there is a need to provide acalibration method and a measurement tool that make it possible tocalculate a high-accuracy calibration parameter for calibrating a stereocamera.

SUMMARY OF THE INVENTION

A calibration method calibrates a stereo camera. The calibration methodincludes: measuring a relative position between the stereo camera and anobject that is placed so as to fall within an image capturing area ofthe stereo camera; acquiring a captured image that is captured by thestereo camera and includes the object; and determining a calibrationparameter for calibrating the stereo camera based on the relativeposition and the captured image.

A calibration method calibrates a stereo camera by using a measurementtool that includes a first member having a surface that includes a chartthat is used to calibrate the stereo camera, a light source that isinstalled on the surface and that emits light with a uniform intensityregardless of a position on the surface, and a second member that coversthe light source and emits the light through a plurality of holes. Thecalibration method includes: acquiring a captured image that is capturedby the stereo camera and includes the measurement tool as an object;measuring a deviation of a direction of the measurement tool from afacing position of the stereo camera based on a position of maximumbrightness in the captured image; and determining a calibrationparameter for calibrating the stereo camera based on the deviation ofthe direction of the measurement tool and the captured image.

A measurement tool includes: a first member that has a surface thatincludes a chart that is used to calibrate the stereo camera; a lightsource that is installed on the surface and that emits light with auniform intensity regardless of a position on the surface; and a secondmember that covers the light source and emits the light through aplurality of holes.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram that illustrates an example of the relationshipamong a measurement tool, a stereo camera, and a calibration deviceaccording to a first embodiment.

FIG. 2 is a diagram that illustrates an example of the configuration ofthe stereo camera according to the first embodiment.

FIG. 3 is a diagram that illustrates a distance measurement principlethat uses the stereo camera.

FIG. 4 is a diagram that illustrates an example of the measurement toolaccording to the first embodiment.

FIG. 5 is a diagram that illustrates an example of a distancemeasurement device according to the first embodiment.

FIG. 6 is a diagram that illustrates an example of the configuration ofthe calibration device according to the first embodiment.

FIG. 7 is a diagram that illustrates an example of an object coordinatesystem according to the first embodiment.

FIG. 8 is a diagram that illustrates an example of a method fordetermining the three-dimensional coordinates that indicate the positionof a first camera according to the first embodiment.

FIG. 9 is a diagram that illustrates an example of a camera coordinatesystem of the first camera according to the first embodiment.

FIG. 10 is an overall schematic flowchart of a calibration methodaccording to the first embodiment.

FIG. 11 is a flowchart that illustrates an example of the calibrationmethod for the calibration device according to the first embodiment.

FIG. 12 is a flowchart that illustrates an example of the overall flowof the calibration method according to the first embodiment.

FIG. 13 is a diagram that illustrates an example of the case where thedistance measurement device according to a modified example of the firstembodiment measures the distance to the first camera (a second camera)by using an intermediate measurement point.

FIG. 14 is a diagram that illustrates an example of the relationshipamong a measurement tool of a second embodiment, the stereo camera, andthe calibration device.

FIG. 15A is a cross-sectional view that illustrates a cross-sectionalsurface of an angle measurement plate according to the secondembodiment.

FIG. 15B is a front view that illustrates the front surface of the anglemeasurement plate according to the second embodiment.

FIG. 16 is a front view of a first member according to the secondembodiment.

FIG. 17 is a diagram that illustrates a case where a hole 106, a hole107, and a hole 108 that are formed on a second member are seen from theposition of the optical center of the first camera.

FIG. 18A is a diagram that illustrates the shape of the hole 106 whenthe hole 106 is seen from the position of the optical center of thefirst camera.

FIG. 18B is a diagram that illustrates the shape of the hole 107 whenthe hole 107 is seen from the position of the optical center of thefirst camera.

FIG. 18C is a diagram that illustrates the shape of the hole 108 whenthe hole 108 is seen from the position of the optical center of thefirst camera.

FIG. 19A is a diagram that illustrates an image of the angle measurementplate in a case where optical unsharpness is not included.

FIG. 19B is a diagram that illustrates an image of the angle measurementplate in a case where optical unsharpness is included.

FIG. 20 is a diagram that illustrates the measurement of the radius ofthe foot of the mountain of brightness in FIG. 19B.

FIG. 21 is a diagram that illustrates the relationship between theposition of the brightness peak of an image capturing surface and thetilt of the angle measurement plate.

FIG. 22 is a diagram that illustrates the method for determining theequation of a plane that indicates the position of the measurement tool.

FIG. 23 is a diagram that illustrates an example of the configuration ofthe calibration device according to the second embodiment.

FIG. 24 is an overall schematic flowchart of the calibration methodaccording to the second embodiment.

FIG. 25 is a flowchart that illustrates an example of the calibrationmethod for the calibration device according to the second embodiment.

FIG. 26 is a cross-sectional view that illustrates a cross-sectionalsurface of an angle measurement plate according to a third embodiment.

FIG. 27 is a diagram that illustrates the angle at which light isrefracted.

FIG. 28 is a diagram that illustrates light that is emitted through theangle measurement plate according to the third embodiment.

FIG. 29 is a diagram that illustrates the relationship between a tilt ofthe measurement tool and a deviation of the position of the brightnesspeak.

FIG. 30 is a diagram that illustrates the relationship between theperiod of moire and the moving range of the brightness peak.

FIG. 31 is a diagram that illustrates the position of the brightnesspeak due to an adjacent hole.

FIG. 32 is a diagram that illustrates an example of the hardwareconfiguration of the stereo camera and the calibration device accordingto the first to the third embodiments.

DESCRIPTION OF EMBODIMENTS

A detailed explanation is given below, with reference to the attacheddrawings, of an embodiment of a calibration method and a measurementtool.

First Embodiment

FIG. 1 is a diagram that illustrates an example of the relationshipamong a measurement tool 20, a stereo camera 10, and a calibrationdevice 30 according to a first embodiment. FIG. 1 is an example of thecase of a calibration of a captured image that is captured by the stereocamera 10 (car-mounted stereo camera) that is installed inside the frontwindshield of a car. The measurement tool 20 is installed such that itfalls within the image capturing area of the stereo camera 10. Forexample, the measurement tool 20 is installed at a distance of about 2 mfrom the stereo camera 10 such that they almost face each other. Themeasurement tool 20 is used to acquire measurement data for determininga calibration parameter to calibrate the stereo camera 10. Measurementdata is input to a computer that is the calibration device 30, and acalibration parameter is determined by the computer. First, anexplanation is given of the stereo camera 10 that is the target to becalibrated.

FIG. 2 is a diagram that illustrates an example of the configuration ofthe stereo camera 10 according to the first embodiment. The stereocamera 10 according to the present embodiment includes a first camera 1,a second camera 2, a storage unit 3, an external I/F 4, a correctingunit 5, and a calculating unit 6. The first camera 1 captures an imageof the object so as to acquire a first captured image. The second camera2 captures an image of the object so as to acquire a second capturedimage. The first camera 1 and the second camera 2 are arranged inparallel so that their optical axes are parallel to each other. Theimage capturing timings of the first camera 1 and the second camera 2are synchronized, and they simultaneously capture an image of the sameobject.

The storage unit 3 stores the first captured image, the second capturedimage, and a calibration parameter. A calibration parameter is aparameter that is used for correcting a distortion of the first capturedimage and the second captured image. A calibration parameter isdetermined by using a calibration method according to the presentembodiment. The external I/F 4 is an interface for inputting andoutputting data of the storage unit 3. A calibration parameter that isused by the stereo camera 10 is determined by using the calibrationmethod according to the present embodiment, and it is stored in thestorage unit 3 by using the external I/F 4.

The correcting unit 5 reads, from the storage unit 3, the first capturedimage, the second captured image, and the calibration parameter. Thecorrecting unit 5 corrects the first captured image and the secondcaptured image by using an image correction equation that corresponds tothe calibration parameter. The image correction equation is the equationfor correcting the first captured image (the second captured image) byconverting the coordinates of the first captured image (the secondcaptured image). For example, in the case where the coordinates of thefirst captured image (the second captured image) are corrected by anaffine transformation, as the image correction equation can berepresented by using a matrix, a calibration parameter is components ofthe matrix. Furthermore, if the coordinates of the first captured image(the second captured image) are corrected by a non-lineartransformation, a calibration parameter is coefficients of a polynomial,or the like, that represents the conversion. Moreover, the correctingunit 5 may correct any one of the first captured image and the secondcaptured image. Specifically, the image correction equation may be animage correction equation for correcting any one of the captured imagesby using the other one of the captured images as a reference. Thecorrecting unit 5 inputs the corrected first captured image and thecorrected second captured image to the calculating unit 6.

The calculating unit 6 calculates a parallax of each object from thecorrected first captured image and the corrected second captured image.Here, an explanation is given of the parallax and the distancemeasurement principle that uses the parallax.

FIG. 3 is a diagram that illustrates the distance measurement principlethat uses the stereo camera 10. In the example of FIG. 3, the firstcamera 1 (a focal length f, an optical center O₀, and an image capturingsurface S₀) is installed such that the Z axis is in the direction of theoptical axis. Furthermore, the second camera 2 (the focal length f, anoptical center O₁, and an image capturing surface S₁) is installed suchthat the Z axis is in the direction of the optical axis. The firstcamera 1 and the second camera 2 are arranged parallel to the X axis atpositions apart from each other by a distance B (base length).Hereafter, the coordinate system of FIG. 3 is referred to as the “cameracoordinate system”. Furthermore, the coordinate system where the opticalcenter of the first camera 1 is a reference is referred to as the“first-camera coordinate system”. Moreover, the coordinate system wherethe optical center of the second camera 2 is a reference is referred toas the “second-camera coordinate system”.

An image of an object A, which is located at a position away from theoptical center O₀ of the first camera 1 by a distance d in the directionof the optical axis, is formed on P₀ that is the intersection pointbetween a straight line A-O₀ and the image capturing surface S₀.Furthermore, with the second camera 2, an image of the same object A isformed on a position P₁ on the image capturing surface S₁.

Here, P₀′ is the intersection point between the image capturing surfaceS₁ and the straight line that passes through the optical center O₁ ofthe second camera 2 and that is parallel to the straight line A-O₀.Furthermore, D is the distance between P₀′ and P₁. The distance Drepresents the degree of positional deviation (parallax) on the imageswhen images of the same object are taken by two cameras. A triangleA-O₀-O₁ and a triangle O₁-P₀′-P₁ are similar. Therefore, the followingEquation (1) is provided.

$\begin{matrix}{d = \frac{Bf}{D}} & (1)\end{matrix}$

Specifically, the distance d to the object A can be determined from thebase length B, the focal length f, and the parallax D. Furthermore, ifthe first camera 1 and the second camera 2 are accurately positioned,the distance d (the distance between the optical center O₀ of the firstcamera 1 and the object A in the direction of the optical axis) that iscalculated by using the first-camera coordinate system is the same asthe distance d (the distance between the optical center O₁ of the secondcamera 2 and the object A in the direction of the optical axis) that iscalculated by using the second-camera coordinate system.

The foregoing is the distance measurement principle by the stereo camera10. To obtain the distance d to the object A with accuracy, the firstcamera 1 and the second camera 2 need to be accurately positioned.However, there is a possibility that the first camera 1 (the secondcamera 2) is displaced in such a direction that it rotates around the Xaxis, the Y axis, or the Z axis. This causes the coordinates of thefirst captured image (the second captured image) to be displacedsubstantially in an upward, downward, leftward, or rightward direction.Furthermore, in the case of a car-mounted stereo camera that captures animage of the object through the front windshield, a distortion occurs inthe first captured image (the second captured image) due to an effect ofthe front windshield. The stereo camera 10 corrects the first capturedimage (the second captured image) by signal processing using acalibration parameter for correcting an error of a parallax that iscaused by a displacement of the first captured image (the secondcaptured image) due to an assembly tolerance of two cameras and by adistortion of the first captured image (the second captured image) dueto the front windshield.

Returning to FIG. 2, the calculating unit 6 generates a parallax imagethat represents a parallax on a pixel by pixel basis by using thedensity value of a pixel of the captured image (the first captured imageor the second captured image) that is used as a reference to calculatethe parallax. Furthermore, the calculating unit 6 calculates thedistance to the object by using the parallax image and Equation (1).

Next, an explanation is given of the measurement tool 20. FIG. 4 is adiagram that illustrates an example of the measurement tool 20 accordingto the first embodiment. The measurement tool 20 according to thepresent embodiment has a structure like a square plate. Furthermore, theshape and the material of the measurement tool 20 may be optional.Specifically, the measurement tool 20 may be any member that has an areafor acquiring data that is used for a calibration. The surface of themeasurement tool 20 includes five marks 21. The mark 21 is used as ameasurement chart to calculate a parallax. Furthermore, the shape, thenumber, and the position of the marks 21 are not limited to theconfiguration according to the present embodiment and may be optional.Furthermore, the surface of the measurement tool 20 has a shadingpattern that makes it easier to detect a corresponding point that is inthe second captured image and that corresponds to a point in the firstcaptured image. Moreover, a distance measurement device 22 a, a distancemeasurement device 22 b, a distance measurement device 22 c, and adistance measurement device 22 d are provided on the four corners of themeasurement tool 20 like a square plate. Hereafter, the distancemeasurement device 22 a, the distance measurement device 22 b, thedistance measurement device 22 c, and the distance measurement device 22d are simply referred to as the distance measurement device 22 if theyare not to be discriminated from one another.

FIG. 5 is a diagram that illustrates an example of the distancemeasurement device 22 according to the first embodiment. The distancemeasurement device 22 has a biaxially rotatable holding mechanism thatis rotatable in upward, downward, leftward, and rightward directionsaround a measurement point 23 that is previously set. The distancemeasurement device 22 according to the present embodiment measures adistance by using TOF (Time of Flight) of laser light 24. Furthermore, adistance measurement method of the distance measurement device 22 may beoptional. For example, the distance measurement device 22 may measure adistance by using ultrasonic waves.

The distance measurement device 22 acquires the distance information(hereafter, referred to as the “first distance information”) thatindicates the distance to the optical center O₀ (see FIG. 3) of thefirst camera 1 and the distance information (hereafter, referred to asthe “second distance information”) that indicates the distance to theoptical center O₁ (see FIG. 3) of the second camera 2. Furthermore, inFIG. 4, the reason why the distance measurement devices 22 are providedon the four corners of the measurement tool 20 is that the measurementpoints 23 of FIG. 5 are located as far as possible from one another.Thus, it is possible to obtain multiple pieces of first distanceinformation (second distance information) that have values as differentas possible, and it is possible to improve the accuracy with which thecalibration device 30, which is described below, calculates the distancebetween the first camera 1 (the second camera 2) and the measurementtool 20 (the distance in the direction of the optical axis of the firstcamera 1 or the distance in the direction of the optical axis of thesecond camera 2). Furthermore, the number and the position of thedistance measurement devices 22 are not limited to the configurationaccording to the present embodiment and may be optional.

FIG. 6 is a diagram that illustrates an example of the configuration ofthe calibration device 30 according to the first embodiment. Thecalibration device 30 according to the present embodiment includes areceiving unit 31, a first-camera position calculating unit 32, afirst-camera direction calculating unit 33, a second-camera positioncalculating unit 34, a second-camera direction calculating unit 35, adistance calculating unit 36, an ideal-parallax calculating unit 37, aparallax calculating unit 38, and a determining unit 39. The calibrationdevice 30 is an information processing apparatus (computer).

The receiving unit 31 receives multiple (four in the present embodiment)pieces of first distance information, multiple (four in the presentembodiment) pieces of second distance information, the first capturedimage that includes the measurement tool 20 as an object, the secondcaptured image that includes the measurement tool 20 as an object, thethree-dimensional coordinate information on the multiple (five in thepresent embodiment) marks 21 in the object coordinate system, and thethree-dimensional coordinate information on the distance measurementdevice 22 (the measurement point 23) in the object coordinate system.For example, in accordance with a user's operation on the calibrationdevice 30, the receiving unit 31 receives an input that indicates themultiple pieces of first distance information, the first captured image,the second captured image, the three-dimensional coordinate informationon the marks 21, and the three-dimensional coordinate information on thedistance measurement device 22. Here, an explanation is given of theobject coordinate system.

FIG. 7 is a diagram that illustrates an example of the object coordinatesystem according to the first embodiment. FIG. 7 is an example of thecase where the origin of the three-dimensional coordinates is located onthe extreme lower left of the measurement tool 20. It is possible toaccurately obtain the four three-dimensional coordinates that indicatethe positions of the distance measurement device 22 a (the measurementpoint 23 a), the distance measurement device 22 b (the measurement point23 b), the distance measurement device 22 c (the measurement point 23c), and the distance measurement device 22 d (the measurement point 23d) in the object coordinate system. That is, the four three-dimensionalcoordinates in the object coordinate system are already known.Furthermore, the three-dimensional coordinate information on themultiple (five in the present embodiment) marks 21 in the objectcoordinate system is also already known.

Returning to FIG. 6, the receiving unit 31 inputs, to the distancecalculating unit 36, the three-dimensional coordinate information on themultiple (five in the present embodiment) marks 21 in the objectcoordinate system. The receiving unit 31 inputs the first captured imageand the second captured image to the parallax calculating unit 38.

Furthermore, the receiving unit 31 inputs, to the first-camera positioncalculating unit 32, the first distance information and thethree-dimensional coordinate information on the distance measurementdevice 22 (the measurement point 23) in the object coordinate system.Moreover, the receiving unit 31 inputs, to the second-camera positioncalculating unit 34, the second distance information and thethree-dimensional coordinate information on the distance measurementdevice 22 (the measurement point 23) in the object coordinate system.

Furthermore, the receiving unit 31 inputs, to the first-camera directioncalculating unit 33, the first captured image and the three-dimensionalcoordinate information on the distance measurement device 22 (themeasurement point 23) in the object coordinate system. Furthermore, thereceiving unit 31 inputs, to the second-camera direction calculatingunit 35, the second captured image and the three-dimensional coordinateinformation on the distance measurement device 22 (the measurement point23) in the object coordinate system.

On the basis of the multiple (four in the present embodiment) pieces offirst distance information and the three-dimensional coordinateinformation on the distance measurement device 22 (the measurement point23) in the object coordinate system, the first-camera positioncalculating unit 32 calculates the three-dimensional coordinates(hereafter, referred to as the “first camera coordinates”) that indicatethe position of the optical center O₀ of the first camera 1 by using theobject coordinate system.

FIG. 8 is a diagram that illustrates an example of the method fordetermining the three-dimensional coordinates that indicate the positionof the first camera 1 according to the first embodiment. The firstdistance information that is acquired by the distance measurement device22 a is denoted by d0. That is, it denotes the distance from themeasurement point 23 a to the first camera 1. The first distanceinformation that is acquired by the distance measurement device 22 b isdenoted by d1. That is, it denotes the distance from the measurementpoint 23 b to the first camera 1. The first distance information that isacquired by the distance measurement device 22 c is denoted by d2. Thatis, it denotes the distance from the measurement point 23 c to the firstcamera 1.

The three-dimensional coordinates that indicate the position of thefirst camera 1 can be calculated by using the object coordinate systemin theory as described below. First, the measurement point 23 a is setas a center, and a point set 25 a that represents a sphere with a radiusd0 is determined. Next, the measurement point 23 b is set as a center,and a point set 25 b that represents a sphere with a radius d1 isdetermined. Then, the measurement point 23 c is set as a center, and apoint set 25 c that represents a sphere with a radius d2 is determined.Then, the point set that is included in both the point set 25 a and thepoint set 25 b is determined. This point set is d0&d1 of FIG. 8. d0&d1is the point set (circular arc) that is represented by using theintersection point between the point set that represents the sphere withthe radius d0 and the point set that represents the sphere with theradius d1. Then, the point set that is included in both the point set 25b and the point set 25 c is determined. This point set is d1&d2 of FIG.8. d1&d2 is the point set (circular arc) that is represented by usingthe intersection point between the point set that represents the spherewith the radius d1 and the point set that represents the sphere with theradius d2. Finally, an intersection point C between the circular arcthat is represented by d0&d1 of FIG. 8 and the circular arc that isrepresented by d1&d2 is determined, whereby the first camera coordinatescan be calculated. That is, in theory, the first camera coordinates canbe calculated if there are three pieces of first distance information.

However, in consideration of a measurement error of the distancemeasurement device 22, it is preferable to calculate the first cameracoordinates by using the more measurement points 23 (four in the presentembodiment). Therefore, the first-camera position calculating unit 32calculates the intersection point C by performing a least squaresapproximation using, for example, the following Equation (2), therebycalculating the first camera coordinates.

$\begin{matrix}{{E_{1}(p)} = {\sum\limits_{i = 1}^{n}\; \left( {{{{pi} - p}} - {di}} \right)^{2}}} & (2)\end{matrix}$

Here, n is the number of the measurement points 23. pi is thethree-dimensional coordinates of the i-th measurement point 23. di isthe distance from the i-th measurement point 23 to the first camera 1,which is measured by the distance measurement device 22.

Returning to FIG. 6, the first-camera position calculating unit 32inputs, to the first-camera direction calculating unit 33, the firstcamera coordinates that are calculated by using the object coordinatesystem. Furthermore, the first-camera position calculating unit 32inputs, to the distance calculating unit 36, the first cameracoordinates that are calculated by using the object coordinate system.

The first-camera direction calculating unit 33 receives, from thereceiving unit 31, the first captured image and the three-dimensionalcoordinate information on the distance measurement device 22 (themeasurement point 23) in the object coordinate system. Furthermore, thefirst-camera direction calculating unit 33 receives, from thefirst-camera position calculating unit 32, the first camera coordinatesthat are calculated by using the object coordinate system.

The first-camera direction calculating unit 33 uses the cameracoordinate system to calculate the direction (the direction of theoptical axis) of the first camera 1 on the basis of thethree-dimensional coordinates of the measurement point 23 (23 a, 23 b,23 c, 23 d), the two-dimensional coordinates of the image of themeasurement point 23 (23 a, 23 b, 23 c, 23 d) in the first capturedimage, and the focal length of the first camera 1. Specifically, thefirst-camera direction calculating unit 33 first converts thethree-dimensional coordinates of the measurement point 23 that iscalculated by using the object coordinate system into the cameracoordinate system where its origin is the optical center O₀ of the firstcamera 1. Specifically, this camera coordinate system is the coordinateswhere, in a pinhole camera model, the position of a pinhole is theorigin. Furthermore, at this time, the camera coordinate system does notstill conform to the direction of the camera coordinate system that iscaused due to a deviation of the direction of the first camera. Next,the first-camera direction calculating unit 33 calculates the triaxialrotation angle r=(α1, β1, γ1) to make the camera coordinate systemconform to the direction of the camera coordinate system that is causeddue to a deviation of the direction of the first camera.

FIG. 9 is a diagram that illustrates an example of the camera coordinatesystem of the first camera 1 according to the first embodiment. Thethree-dimensional coordinates of the measurement point 23 (object) are(x, y, z), the two-dimensional coordinates on an image capturing surface40 are (u, v), and the focal length of the first camera 1 is f. Here,the position of an image 41 of the measurement point 23 on the imagecapturing surface 40 can be represented by using the following Equation(3).

$\begin{matrix}{\left( {u,v} \right) = \left( {{f\frac{x}{z}},{f\frac{y}{z}}} \right)} & (3)\end{matrix}$

Generally, the two-dimensional coordinates (u, v) on the image capturingsurface 40 can be calculated by using Equation (3) on the basis of theposition of the optical center of the camera, the focal length f of thecamera, the three-dimensional coordinates p=(x, y, z) of the measurementpoint 23, and the camera direction (triaxial rotation angle r=(α1, β1,γ1)). Furthermore, α1 denotes the rotation angle with respect to the Xaxis, β1 denotes the rotation angle with respect to the Y axis, and γ1denotes the rotation angle with respect to the Z axis.

Conversely, the camera direction (triaxial rotation angle r=(α1, β1,γ1)) can be determined by using Equation (3) on the basis of theposition of the optical center of the camera, the focal length f of thecamera, the three-dimensional coordinates p=(x, y, z) of the measurementpoint 23, and the two-dimensional coordinates (u, v) on the imagecapturing surface 40.

The function F for calculating the two-dimensional coordinates (u, v) onthe image capturing surface 40 is obtained by using the relation ofEquation (3) on the basis of the triaxial rotation angle r=(α1, β1, γ1)and the three-dimensional coordinates p=(x, y, z) of the measurementpoint 23 ((u, v)=F(r, p)).

The first-camera direction calculating unit 33 performs a least squaresapproximation by using the following Equation (4), thereby calculatingthe triaxial rotation angle r=(α1, β1, γ1).

$\begin{matrix}{{E_{2}(r)} = {\sum\limits_{i = 1}^{n}\; {{{F\left( {r,{pi}} \right)} - \left( {{ui},{vi}} \right)}}^{2}}} & (4)\end{matrix}$

Here, n is the number of the measurement points 23. pi is thethree-dimensional coordinates of the i-th measurement point 23. (ui, vi)is the two-dimensional coordinates that correspond to the i-thmeasurement point 23 and that is on the image capturing surface 40.

As the camera direction (the triaxial rotation angle r) has threevariables, if the two-dimensional coordinates of two points on the imagecapturing surface 40 are present as a restriction condition, the cameradirection can be determined by using Equation (3). The reason why thefirst-camera direction calculating unit 33 calculates the triaxialrotation angle r=(α1, β1, γ1) by using Equation (4) is that the firstcaptured image is captured through the front windshield. Specifically,as the first captured image has a distortion that is caused due to thefront windshield, it is preferable to calculate the triaxial rotationangle r=(α1, β1, γ1) by using a large number of the measurement points23 and by performing a least squares approximation using Equation (4).

Furthermore, if a distortion is different depending on (u, v), and, forexample, if it is previously predicted that a peripheral section of theimage is largely distorted compared to the middle of the screen, theposition of the measurement point may be accordingly located such thatthe measurement point 23 appears in the middle section of the firstcaptured image. Moreover, in Equation (4), weighting may be applieddepending on the measurement point 23.

Furthermore, the measurement point 23 of the distance measurement device22 is used as the measurement point 23; however, the arbitrarymeasurement point 23 may be used if its coordinates are known in theobject coordinate system. For example, the arbitrary measurement point23 that is on the measurement tool 20 and that is suitable formeasurement or the arbitrary measurement point 23 that is not on themeasurement tool 20 and that is suitable for measurement may be used.

Returning to FIG. 6, the first-camera direction calculating unit 33inputs, to the distance calculating unit 36, the direction (triaxialrotation angle r=(α1, β1, γ1)) of the first camera 1 that is calculatedby using the above-described Equation (4).

The second-camera position calculating unit 34 calculates thethree-dimensional coordinates (hereinafter, referred to as the “secondcamera coordinate”) that indicate the position of the optical center O₁of the second camera 2 by using the object coordinate system on thebasis of multiple (four in the present embodiment) pieces of seconddistance information. As the method for calculating the second cameracoordinates is the same as the method for calculating the first cameracoordinates, its detailed explanation is omitted. The second-cameraposition calculating unit 34 inputs, to the second-camera directioncalculating unit 35, the second camera coordinates that are calculatedby using the object coordinate system. Furthermore, the second-cameraposition calculating unit 34 inputs, to the parallax calculating unit38, the second camera coordinates that are calculated by using theobject coordinate system.

The second-camera direction calculating unit 35 receives, from thereceiving unit 31, the second captured image and the three-dimensionalcoordinate information on the distance measurement device 22 (themeasurement point 23) in the object coordinate system. Moreover, thesecond-camera direction calculating unit 35 receives, from thesecond-camera position calculating unit 34, the second cameracoordinates that are calculated by using the object coordinate system.

The second-camera direction calculating unit 35 uses the cameracoordinate system to calculate the direction (the direction of theoptical axis) of the second camera 2 on the basis of thethree-dimensional coordinates of the measurement point 23 (23 a, 23 b,23 c, 23 d), the two-dimensional coordinates of the image of themeasurement point 23 (23 a, 23 b, 23 c, 23 d) in the second capturedimage, and the focal length of the second camera 2. As the method forcalculating the direction of the second camera 2 is the same as themethod for calculating the direction of the first camera 1, its detailedexplanation is omitted. The second-camera direction calculating unit 35inputs, to the distance calculating unit 36, the direction (triaxialrotation angle r=(α2, β2, γ2)) of the second camera 2.

The distance calculating unit 36 receives, from the receiving unit 31,the three-dimensional coordinate information on the marks 21 in theobject coordinate system. Furthermore, the distance calculating unit 36receives, from the first-camera position calculating unit 32, the firstcamera coordinates that are calculated by using the object coordinatesystem. Furthermore, the distance calculating unit 36 receives thedirection (triaxial rotation angle r=(α1, β1, γ1)) of the first camera 1from the first-camera direction calculating unit 33. Furthermore, thedistance calculating unit 36 receives, from the second-camera positioncalculating unit 34, the second camera coordinates that are calculatedby using the object coordinate system. Moreover, the distancecalculating unit 36 receives the direction (triaxial rotation angler=(α2, β2, γ2)) of the second camera 2 from the second-camera directioncalculating unit 35.

With respect to each of the marks 21, the distance calculating unit 36calculates the distances d between the optical center O₀ of the firstcamera 1 and the marks 21 in the direction of the optical axis of thefirst camera 1 by using the first-camera coordinate system based on thefirst camera coordinates and the direction of the first camera 1.

Specifically, the distance calculating unit 36 first rotates the cameracoordinate system, in which the first camera coordinates are the origin,based on the direction (triaxial rotation angle r=(α1, β1, γ1)) of thefirst camera 1, thereby converting it into the first-camera coordinatesystem. That is, the first-camera coordinate system is the coordinatesystem where the first camera coordinates are the origin, the directionof the optical axis of the first camera 1 is the Z axis, the straightline that passes through the origin in a horizontal direction on theplane that is vertical to the Z axis that includes the origin is the Xaxis, and the straight line that passes through the origin in a verticaldirection on the plane that is vertical to the Z axis that includes theorigin is the Y axis. Next, the distance calculating unit 36 uses thefirst-camera coordinate system and Equation (1) to calculate thedistances d between the optical center O₀ of the first camera 1 and themarks 21 in the direction of the optical axis of the first camera 1 (seeFIG. 3). Thus, the relative position between the object (the marks 21)and the stereo camera 10 (the optical center O₀ of the first camera 1)is made definite. The distance calculating unit 36 inputs, to theideal-parallax calculating unit 37, the distance information thatindicates the distance d between each of the marks 21 and the stereocamera 10.

Furthermore, the distance calculating unit 36 may calculate thedistances d between the optical center O₁ of the second camera 2 and themarks 21 in the direction of the optical axis of the second camera 2 byusing the second-camera coordinate system based on the second cameracoordinates and the direction of the second camera 2.

The ideal-parallax calculating unit 37 receives the above-describeddistance information from the distance calculating unit 36. Based on theabove-described distance information, the ideal-parallax calculatingunit 37 calculates ideal parallaxes that indicate the ideal parallaxesbetween the marks 21 included in the first captured image and the marks21 included in the second captured image by using Equation (1) withrespect to each of the marks 21. The ideal-parallax calculating unit 37inputs the ideal parallaxes to the determining unit 39.

The parallax calculating unit 38 receives the first captured image andthe second captured image from the receiving unit 31. The parallaxcalculating unit 38 calculates, with respect to each of the marks 21,the parallaxes between the marks 21 included in the first captured imageand the marks 21 included in the second captured image. The parallaxcalculating unit 38 inputs the parallaxes to the determining unit 39.

The determining unit 39 receives the ideal parallaxes from theideal-parallax calculating unit 37 and receives the parallaxes from theparallax calculating unit 38. Furthermore, the determining unit 39receives the first captured image and the second captured image from thereceiving unit 31. The determining unit 39 determines a calibrationparameter for correcting the first captured image and the secondcaptured image on the basis of the parallaxes and the ideal parallaxes.For example, the determining unit 39 determines a calibration parameterfor a correction such that the difference between the parallaxes and theideal parallaxes becomes zero.

Furthermore, if a parallax occurs in a vertical direction (the Y-axisdirection), the determining unit 39 determines a calibration parameterfor a correction such that the parallax in the vertical directionbecomes zero regardless of the distance to the object (multiple marks).The reason for this is that it is assumed that a parallax occurs only ina horizontal direction (the X-axis direction).

FIG. 10 is an overall schematic flowchart of a calibration methodaccording to the first embodiment. The stereo camera 10 acquires thecaptured image (Step S101). Specifically, the first camera 1 acquiresthe first captured image, and the second camera 2 acquires the secondcaptured image.

Next, the calibration device 30 measures the relative position betweenthe stereo camera 10 and the object (Step S102). Specifically, thecalibration device 30 measures the relative positions between theoptical center O₀ of the first camera 1 of the stereo camera 10 and themarks 21 on the measurement tool 20 by performing Step S6 to Step S10that are described below.

Next, the calibration device 30 determines a calibration parameter basedon the relative position (Step S103). Specifically, the calibrationdevice 30 determines a calibration parameter for correcting at least oneof the first captured image and the second captured image such that theparallaxes between the marks 21 included in the first captured image andthe marks 21 included in the second captured image matches idealparallaxes that indicates the ideal parallaxes based on the relativepositions that are measured at Step S102.

Next, a detailed explanation is given of a calibration method of thecalibration device 30 according to the present embodiment with referenceto the flowchart. FIG. 11 is a flowchart that illustrates an example ofthe calibration method for the calibration device 30 according to thefirst embodiment.

The receiving unit 31 receives the coordinate information on themeasurement tool 20 (Step S1). The coordinate information is thethree-dimensional coordinate information on the multiple (five in thepresent embodiment) marks 21 in the object coordinate system and thethree-dimensional coordinate information on the multiple (four in thepresent embodiment) distance measurement devices 22 (the measurementpoints 23) in the object coordinate system. Furthermore, the receivingunit 31 receives multiple (four in the present embodiment) pieces offirst distance information (Step S2). Furthermore, the receiving unit 31receives multiple (four in the present embodiment) pieces of seconddistance information (Step S3). Furthermore, the receiving unit 31receives the first captured image that includes the measurement tool 20as an object (Step S4). Moreover, the receiving unit 31 receives thesecond captured image that includes the measurement tool 20 as an object(Step S5).

Next, the first-camera position calculating unit 32 calculates the firstcamera coordinates that indicate the position of the optical center O₀of the first camera 1 by using the object coordinate system on the basisof the multiple pieces of first distance information and thethree-dimensional coordinate information on the distance measurementdevice 22 (the measurement point 23) in the object coordinate system(Step S6).

Next, the second-camera position calculating unit 34 calculates thesecond camera coordinates that indicate the position of the opticalcenter O₁ of the second camera 2 by using the object coordinate systemon the basis of the multiple pieces of second distance information andthe three-dimensional coordinate information on the distance measurementdevice 22 (the measurement point 23) in the object coordinate system(Step S7).

Next, the first-camera direction calculating unit 33 uses the cameracoordinate system to calculate the direction (the direction of theoptical axis) of the first camera 1 on the basis of thethree-dimensional coordinates of the measurement point 23 (23 a, 23 b,23 c, 23 d), the two-dimensional coordinates of the image of themeasurement point 23 in the first captured image, and the focal lengthof the first camera 1 (Step S8).

Next, the second-camera direction calculating unit 35 uses the cameracoordinate system to calculate the direction (the direction of theoptical axis) of the second camera 2 on the basis of thethree-dimensional coordinates of the measurement point 23 (23 a, 23 b,23 c, 23 d), the two-dimensional coordinates of the image of themeasurement point 23 in the second captured image, and the focal lengthof the second camera 2 (Step S9).

Next, with respect to each of the marks 21, the distance calculatingunit 36 uses the first-camera coordinate system that is based on thefirst camera coordinates and the direction of the first camera 1 tocalculate the distances d between the optical center O₀ of the firstcamera 1 and the marks 21 in the direction of the optical axis of thefirst camera 1 (Step S10). At Step S6 to Step S10, the relative positionbetween the object (the marks 21) and the stereo camera 10 (the opticalcenter O₀ of the first camera 1) is made definite.

Next, based on the distances d in the direction of the optical axis ofthe first camera 1, which is calculated at Step S10, the ideal-parallaxcalculating unit 37 calculates ideal parallaxes that indicates the idealparallaxes between the marks 21 included in the first captured image andthe marks 21 included in the second captured image by using Equation (1)with respect to each of the marks 21 (Step S11).

Next, the parallax calculating unit 38 calculates, with respect to eachof the marks 21, the parallaxes between the marks 21 included in thefirst captured image and the marks 21 included in the second capturedimage (Step S12).

Next, the determining unit 39 determines a calibration parameter forcorrecting the first captured image and the second captured image suchthat the differences between the parallaxes and the ideal parallaxesbecome zero (Step S13).

Furthermore, at the above-described Step S10, the distance calculatingunit 36 may use the second-camera coordinate system that is based on thesecond camera coordinates and the direction of the second camera 2 tocalculate the distances d between the optical center O₁ of the secondcamera 2 and the marks 21 in the direction of the optical axis of thesecond camera 2.

Next, an explanation is given of the overall flow of the calibrationmethod that uses the above-described measurement tool 20 and theabove-described calibration device 30 according to the presentembodiment. FIG. 12 is a flowchart that illustrates an example of theoverall flow of the calibration method according to the firstembodiment.

First, the measurement tool 20 is installed in front of a vehicle wherethe stereo camera 10 is mounted such that they almost face each other(Step S21). Next, the measurement tool 20 measures the distances betweenthe optical center O₀ of the stereo camera 10 and the measurement points23 on the measurement tool 20 (Step S22). Specifically, the distancemeasurement devices 22 on the four corners of the measurement tool 20are appropriately rotated, and the distance to the optical center O₀ ofthe stereo camera 10 is measured through the front windshield of thevehicle. Next, the stereo camera 10 captures an image of the measurementtool 20 through the front windshield without changing the position ofthe measurement tool 20 (Step S23).

Next, the measurement data that is measured at Step S22 is input to thecalibration device 30 (Step S24). At this time, the coordinates that arein the object coordinate system and that indicate the marks 21 on themeasurement tool 20 and the coordinates that are in the objectcoordinate system and that indicate the measurement points 23 on themeasurement tool 20 are simultaneously input. Next, the captured imagethat is captured at Step S23 is input to the calibration device 30 (StepS25).

Next, the calibration device 30 calculates the relative position betweenthe measurement tool 20 and the optical center O₀ of the stereo camera10 (Step S26). Specifically, the calibration device 30 calculates theposition and the direction of the stereo camera 10 (Step S6 to Step S9of FIG. 11). Then, the calibration device 30 calculates the distance dbetween the measurement tool 20 (the marks 21) and the optical center O₀of the stereo camera 10 in the direction of the optical axis of thefirst camera 1 (Step S10 of FIG. 11).

Next, the calibration device 30 calculates the ideal parallaxes on thebasis of the relative position that is calculated at Step S26 (StepS27). Specifically, the calibration device 30 calculates the idealparallaxes by using the method of Step S11 in FIG. 11. Then, thecalibration device 30 calculates parallaxes on the basis of the capturedimage that is input at Step S25 (Step S28). Specifically, thecalibration device 30 calculates parallaxes by using the method of StepS12 in FIG. 11. Then, the calibration device 30 determines a calibrationparameter for calibrating the stereo camera 10 on the basis of therelative position and the captured image (Step S29). Specifically, thecalibration device 30 determines a calibration parameter for calibratingthe captured image that is captured by the stereo camera 10 on the basisof the ideal parallaxes that are calculated from the relative positionand the parallaxes that are calculated from the captured image (Step S13of FIG. 11).

As described above, with the calibration method according to the firstembodiment, the relative position between the stereo camera 10 and themeasurement tool 20 that is installed such that it falls within theimage capturing area of the stereo camera 10 is measured, and acalibration parameter for calibrating the stereo camera 10 is determinedon the basis of the relative position and the captured image that iscaptured by the stereo camera 10 and that includes the measurement tool20 as an object. Thus, it is possible to calculate a calibrationparameter with high accuracy with respect to the stereo camera 10 thatis installed in a vehicle for which it is difficult to ensure theinstallation location accuracy.

Furthermore, in the first embodiment, an explanation is given of a casewhere a calibration is performed on the stereo camera 10 that isinstalled in a car. However, the calibration method according to thefirst embodiment may be applied to the stereo camera 10 that is notlimited to be installed in a vehicle (movable object), such as a car,but may be installed in any object. Moreover, if a calibration needs tobe performed on the stereo camera 10 with higher accuracy even if thestereo camera 10 is not installed on an object, the method according tothe present embodiment may be applied.

Modified Example of the First Embodiment

Next, an explanation is given of a modified example of the calibrationmethod according to the first embodiment. With the calibration methodaccording to the modified example of the first embodiment, the distancemeasurement device 22 does not measure the distance to the opticalcenter of the first camera 1 (the second camera 2) but measures thedistance to an intermediate measurement point that is located in themiddle with respect to the optical center of the first camera 1 (thesecond camera 2). This is because it is difficult to directly measurethe optical center of the camera as it is generally located inside alens.

FIG. 13 is a diagram that illustrates an example of the case where thedistance measurement device 22 according to the modified example of thefirst embodiment measures the distance to the first camera 1 (the secondcamera 2) by using the intermediate measurement point. FIG. 13 is anexample of the case where an intermediate measurement point 61 islocated in a direction perpendicular to the measurement device 20 and islocated close to a position 62 of the first camera 1 (the second camera2). For example, the intermediate measurement point 61 is located on thefront windshield.

The distance measurement device 22 measures the distance information(hereafter, referred to as the “first intermediate distanceinformation”) that indicates the distance between the measurement point23 on the measurement tool 20 and the intermediate measurement pointthat is provided near the optical center of the first camera 1.Furthermore, the distance measurement device 22 measures the distanceinformation (hereafter, referred to as the “second intermediate distanceinformation”) that indicates the distance between the measurement point23 on the measurement tool 20 and the intermediate measurement pointthat is provided near the optical center of the second camera 2.

The receiving unit 31 receives the first intermediate distanceinformation and the second intermediate distance information.Furthermore, the receiving unit 31 receives the distance from theintermediate measurement point 61 to the optical center of the firstcamera 1 (the second camera 2) as the distance information that isseparately acquired from a measured value, designed value, or the like.

The first-camera position calculating unit 32 (the second-cameraposition calculating unit 34) first determines the coordinates thatindicate the position of the intermediate measurement point 61 by usingEquation (2). Next, the first-camera position calculating unit 32 (thesecond-camera position calculating unit 34) calculates the coordinatesof the optical center of the first camera 1 (the second camera 2) byusing the coordinates that indicate the position of the intermediatemeasurement point 61 and the distance information that indicates thedistance from the intermediate measurement point 61 to the opticalcenter of the first camera 1 (the second camera 2).

Furthermore, the distance from the intermediate measurement point 61 tothe optical center of the first camera 1 (the second camera 2) may beseparately measured by using the camera coordinate system without usingthe object coordinate system of the measurement tool 20. Furthermore, ifthe difference between the direction of a camera optical axis 63 and thedirection of the straight line that is perpendicular to the measurementtool 20 is small, a position 65 that is located on the straight linethat is perpendicular to the measurement tool 20 may be regarded as theposition of the optical center of the first camera 1 (the second camera2). This is because an error 64 from the actual position of the firstcamera 1 (the second camera 2) is vanishingly small.

Furthermore, the distance between the intermediate measurement point 61and the measurement point 23 of the measurement tool 20 may be measuredby using a measuring tape, or the like, without using the distancemeasurement device 22.

Second Embodiment

Next, an explanation is given of a second embodiment. A measurement toolthat is used for a calibration of the stereo camera 10 in the secondembodiment is different from that in the first embodiment. Any one ofthe first camera 1 and the second camera 2 is used for a calibration ofthe stereo camera 10 in the second embodiment. Although an explanationis given in the second embodiment by using the first camera 1, thesecond camera 2 may be used.

FIG. 14 is a diagram that illustrates an example of the relationshipamong a measurement tool 120 of the second embodiment, the stereo camera10, and the calibration device 30. According to the second embodiment,the measurement tool 120 is used instead of the measurement tool 20. Asthe explanations of the stereo camera 10 and the calibration device 30are the same as those in FIG. 1, they are omitted. The measurement tool120 is used to measure the relative position with the stereo camera 10as is the case with the measurement tool 20 of the first embodiment;however, its configuration is different from that of the measurementtool 20 according to the first embodiment. The measurement tool 120according to the second embodiment includes an angle measurement plate101 and a first member 102. The angle measurement plate 101 is used tomeasure the angle that indicates a displacement of the measurement tool120 that is tilted in a horizontal direction and the angle thatindicates a displacement of the measurement tool 120 that is tilted in avertical direction. The first member 102 is used as a chart for acalibration of the stereo camera 10.

A detailed explanation is given, with reference to FIG. 15A to FIG. 16,of a configuration of the measurement tool 120. FIG. 15A is across-sectional view that illustrates a cross-sectional surface of theangle measurement plate 101 according to the second embodiment. FIG. 15Bis a front view that illustrates the front surface of the anglemeasurement plate 101 according to the second embodiment. FIG. 16 is afront view of the first member 102 according to the second embodiment.

The angle measurement plate 101 includes a light source 103 and a secondmember 104. The light source 103 is a plane diffused light source thathas a uniform brightness distribution. Specifically, the light source103 emits light with a uniform intensity regardless of the position onthe surface of the first member 102 (light for which a difference in thelight intensity depending on the position on the surface falls within apredetermined range).

The second member 104 is installed such that it covers the light source103, and light of the light source 103 is emitted through multiple holes105. Each of the holes 105 is formed in a direction perpendicular to thesurface of the first member 102 with a predetermined pitch. In theexample of FIG. 15B, the circular holes 105 with a diameter b are formedsuch that they are aligned with a pitch a in vertical and horizontaldirections. Furthermore, the number, the shape, and the way ofarrangement of the holes 105 are not limited to the configuration ofFIG. 15B and may be optional. Furthermore, the material of the secondmember 104 may be optional. The material of the second member 104 is,for example, a metal.

The angle measurement plate 101 (the second member 104) is located inthe middle of the surface of the first member 102. Furthermore, thefirst member 102 includes a mark 111 on the top of the angle measurementplate 101 (the second member 104). The mark 111 is used as a referencepoint for calculating the distance between the first member 102 (themeasurement tool 120) and the stereo camera 10 (the first camera 1) thatis the target to be calibrated. Furthermore, as is the case with thesurface of the measurement tool 20 according to the first embodiment,the surface of the first member 102 has a shading pattern that makes iteasier to detect a corresponding point that is on the second capturedimage and that corresponds to a point on the first captured image.

FIG. 17 is a diagram that illustrates a case where a hole 106, a hole107, and a hole 108 that are formed on the second member 104 are seenfrom the position of the optical center O₀ of the first camera 1. Withrespect to the position of the hole 106, the line of sight from theposition of the optical center O₀ runs at right angles to the surface ofthe second member 104; therefore, light of the light source 103 that islocated behind the second member 104 looks like the shape of FIG. 18A.With respect to the position of the hole 107, the line of sight from theposition of the optical center O₀ enters the hole 107 of the secondmember 104 at a tilt; therefore, light of the light source 103 that islocated behind the second member 104 looks like the shape of FIG. 18B.With respect to the position of the hole 108, the line of sight from theposition of the optical center O₀ does not enter the hole 108 of thesecond member 104; therefore, light of the light source 103 that islocated behind the second member 104 is invisible (FIG. 18C).

Specifically, if an image of the angle measurement plate 101 (the secondmember 104) is captured by the first camera 1 that has a much higherresolution compared to the pitch a of the holes, the image of the holenear the point where the line of sight from the optical center O₀ of thefirst camera 1 runs at right angles to the surface of the second member104 in the captured image is large. Furthermore, the area of the imageof the hole declines as it is farther away from the point where the lineof sight from the optical center O₀ of the first camera 1 runs at rightangles to the surface of the second member 104, and the image of thehole at a position much farther away is not captured.

Here, an explanation is given of the pitch a of the holes 105 of thesecond member 104. The pitch a of the holes 105 in FIG. 15B is set to belower than the resolution limit of the stereo camera 10 (the firstcamera 1). For example, if the condition is a (half) view angle of 20degrees, an image capturing distance (calibration distance) of 2 m, andthe pixel sensor of 640×480, and if the pitch a is approximately equalto or less than 2 mm, it is smaller than the pixel pitch according tothe following Equation (5).

tan(20[deg])×2000/320≈2.3  (5)

In a case where the pitch a is smaller than the pixel pitch, if an imageof the angle measurement plate 101 (the second member 104) is capturedby the stereo camera 10 (the first camera 1), the captured image looksas in FIG. 19A. However, in actuality, even if the pitch a isapproximately equal to or less than 2 mm, it exceeds the resolutionlimit due to the effect of a pixel aperture feature, unsharpness of animaging forming optical system, or optical unsharpness of an optical LPF(Low Pass Filter), or the like, in the case of a color camera.Therefore, it is difficult to discriminate among the individual holes inthe captured image, and a single large brightness mountain (brightnessdistribution) like FIG. 19B in a form as if the image of FIG. 19A isshaded off is obtained. The top of the mountain of brightness in FIG.19B corresponds to, for example, the vicinity of the hole 106 in FIG.17. Furthermore, the foot of the mountain of brightness in FIG. 19Bcorresponds to, for example, the vicinity of the hole 108 in FIG. 17.

FIG. 20 is a diagram that illustrates the measurement of the radius c ofthe foot of the mountain of brightness in FIG. 19B. It is assumed thatthe thickness of the second member 104 is 40 mm and the diameter of thehole 108 is 1 mm. Furthermore, it is assumed that the (half) view angleof the stereo camera 10 (the first camera 1) is 20 degrees and the pixelsensor has 640×480 pixels. Here, the radius c of the foot of themountain of brightness is about 22 pixels according to the followingEquation (6).

c=f/40×1=320/tan(20[deg])/40≈22  (6)

As described above, instead of the images of the individual holes 105 ofFIG. 15B, the large smoothed continuous brightness mountain isconsidered; therefore, even if the position where the optical center O₀of the stereo camera 10 (the first camera 1) is not right above the hole105 corresponds to a direction perpendicular to the surface of the anglemeasurement plate 101, it can be determined that the position ofbrightness peak indicates a direction perpendicular to the surface ofthe angle measurement plate 101. Thus, regardless of the position of theoptical center O₀ of the stereo camera 10 (the first camera 1), it isdetermined that the position of the brightness peak of the brightnessmountain (brightness distribution) of the captured image by capturing animage of the angle measurement plate 101 indicates a directionperpendicular to the surface of the angle measurement plate 101.

Furthermore, the entire mountain of brightness is approximated by usinga function, such as Gauss function (exp(−r²)), and an averagedistribution of a large number of pixel values is estimated; thus, theeffect of random noise that is included in an individual pixel value canbe reduced, and the position of the brightness peak can be accuratelyestimated.

FIG. 21 is a diagram that illustrates the relationship between theposition of the brightness peak of the image capturing surface 40 andthe tilt of the angle measurement plate 101. The middle of the imagecapturing surface 40 is used as an origin and representation is made inunits of a pixel. Here, the coordinates (i_(p), j_(p)) of the positionof the brightness peak on the image capturing surface 40 (the capturedimage) indicate the position of the foot of the perpendicular line drawnfrom the optical center O₀ of the stereo camera 10 (the first camera 1)to the angle measurement plate 101. Therefore, if the angle measurementplate 101 faces the stereo camera 10 (the first camera 1), i.e., if thenormal to the surface of the measurement tool 120 is parallel to theoptical axis of the stereo camera 10 (the first camera 1), the positionof the brightness peak ought to be in the middle (origin) of thecaptured image. Specifically, if the position (i_(p), j_(p)) of thebrightness peak deviates from the middle (origin) of the captured image,it is possible to determine, from the coordinates (i_(p), j_(p)) thatare on the captured image (the image capturing surface 40) and thatindicate the position of the brightness peak, the angle that indicates adisplacement in a horizontal direction of the measurement tool 120 thatis tilted with respect to a facing direction and the angle thatindicates a displacement in a vertical direction. Specifically, if thefocal length (in units of a pixel) of the stereo camera 10 (the firstcamera 1) is f, the direction of the normal to the angle measurementplate 101 can be determined by using (i_(p), j_(p), f). In other words,it is possible to determine the direction of the stereo camera 10 (thefirst camera 1) that faces the angle measurement plate 101 that isinstalled such that it is tilted with respect to a facing direction.

Next, a detailed explanation is given of a method for determining theequation of a plane that indicates the position of the angle measurementplate 101 (the measurement tool 120). FIG. 22 is a diagram thatillustrates the method for determining the equation of a plane thatindicates the position of the measurement tool 120. The followingEquation (7) represents the equation of the plane that indicates themeasurement tool 120 in the coordinate system that uses the opticalcenter O₀ of the stereo camera 10 (the first camera 1) as an origin.

ax+by+cz+d=0  (7)

As illustrated in FIG. 21, the direction of the normal to the anglemeasurement plate 101 can be represented as (i_(p), j_(p), f).Therefore, as the normal vector of the plane can be determined by using(i_(p), j_(p), f), (a, b, c) (i_(p), j_(p), f). Next, to determine avariable d of the equation of the plane, the mark 111 of the measurementtool 120 (the first member 102) is measured by using a laser distancemeter, or the like, and the distance is defined as d_(c). Furthermore,the coordinates that indicate the position of the mark 111 on thecaptured image are (i_(c), j_(c)). If the focal length (on a pixel topixel basis) of the stereo camera 10 (the first camera 1) is f, thepoint (x_(c), y_(c), z_(c)) by the distance d_(c) in the direction ofthe vector (i_(c), j_(c), f) is the coordinates that indicate theposition of the mark 111. Specifically, the coordinates (x_(c), y_(c),z_(c)) on the plane that indicates the position of the mark 111 can becalculated by using the following Equation (8).

$\begin{matrix}{\left( {x_{c},y_{c},z_{c}} \right) = {\frac{d_{c}}{\sqrt{i_{c}^{2} + j_{c}^{2} + f^{2}}}\left( {i_{c},j_{c},f} \right)}} & (8)\end{matrix}$

Therefore, the variable d of the equation of the plane can be determinedby using the following Equation (9) and the following Equation (10).Thus, the equation (a, b, c, d) of the plane that represents themeasurement tool 120 can be determined.

ax _(c) +by _(c) +cz _(c) +d=0  (9)

d=i _(p) x _(c) −j _(p) y _(c) −fz _(c)  (10)

By using the captured image of the angle measurement plate 101, it ispossible to calculate the angle that indicates a displacement due to atilt of the angle measurement plate 101 in a horizontal direction andthe angle that indicates a displacement due to a tilt in a verticaldirection; however, the distance to the angle measurement plate 101cannot be determined. Therefore, the position of the mark 111 on thecaptured image and the distance information d_(c) to the mark 111 areused in the above explanation. Otherwise, it is possible to use a methodof actually measuring d₁ of FIG. 22, or the like. Furthermore, if thelocation accuracy of the measurement tool 120 is higher (and the angleaccuracy is lower) compared to the acceptable accuracy of a calibrationof the stereo camera 10 (the first camera 1), a fixed value may be usedas the distance information without actually measuring the distancebetween the mark 111 and the stereo camera 10.

Next, an explanation is given of a configuration of the calibrationdevice 30 according to the second embodiment for a calibration of thestereo camera 10 (the first camera 1) by using the above-describedmethod. FIG. 23 is a diagram that illustrates an example of theconfiguration of the calibration device according to the secondembodiment. The calibration device 30 according to the second embodimentincludes a receiving unit 131, a measuring unit 136, an ideal-parallaxcalculating unit 137, a parallax calculating unit 138, and a determiningunit 139.

The receiving unit 131 receives the captured image that is captured bythe stereo camera 10 and includes the measurement tool 120 as an object(the first captured image that is captured by the first camera 1 and thesecond captured image that is captured by the second camera 2).Furthermore, the receiving unit 131 receives the above-describeddistance information d_(c) (see FIG. 22). The receiving unit 131 inputsthe first captured image and the distance information d_(c) to themeasuring unit 136. Furthermore, the receiving unit 131 inputs thecaptured image (the first captured image and the second captured image)to the parallax calculating unit 138.

The measuring unit 136 receives the first captured image and thedistance information d_(c) from the receiving unit 131. The measuringunit 136 determines the direction (normal vector) that is perpendicularto the surface of the measurement tool 120 (the angle measurement plate101) by using the method that is illustrated in FIG. 21 on the basis ofthe position of the maximum brightness of the first captured image.Thus, the measuring unit 136 measures a deviation of the direction ofthe measurement tool 120 (the angle that indicates a displacement due toa tilt in a horizontal direction and the angle that indicates adisplacement due to a tilt in a vertical direction) with respect to afacing position of the stereo camera 10 (the first camera 1).Furthermore, the equation of the plane that indicates the position ofthe measurement tool 120 in the first-camera coordinate system (thecoordinate system in which the optical center O₀ of the first camera 1is an origin) is determined by using the method that is illustrated inFIG. 22 on the basis of the normal vector and the distance informationd_(c). The measuring unit 136 inputs, to the ideal-parallax calculatingunit 137, the information that indicates the equation of the plane.

The ideal-parallax calculating unit 137 receives, from the measuringunit 136, the equation of the plane that indicates the position of themeasurement tool 120. The ideal-parallax calculating unit 137 uses themethod that is illustrated in FIG. 3 to calculate an ideal parallax thatindicates a parallax in the case where an image of the plane that isrepresented by the equation is captured. The ideal-parallax calculatingunit 137 inputs the ideal parallax to the determining unit 139.

The parallax calculating unit 138 receives the captured image (the firstcaptured image and the second captured image) from the receiving unit131. The parallax calculating unit 138 uses a shading pattern of themeasurement tool 120 (the first member 102), or the like, to detect acorresponding point that is in the second captured image and thatcorresponds to a point in the first captured image, thereby calculatinga parallax. The parallax calculating unit 138 inputs the parallax to thedetermining unit 139.

The determining unit 139 receives the ideal parallax from theideal-parallax calculating unit 137 and receives the parallax from theparallax calculating unit 138. Furthermore, the determining unit 139receives the first captured image and the second captured image from thereceiving unit 131. The determining unit 139 determines a calibrationparameter for correcting the first captured image and the secondcaptured image on the basis of the parallax and the ideal parallax. Thedetermining unit 139 determines, for example, a calibration parameterfor correcting at least one of the first captured image and the secondcaptured image such that the difference between the parallax and theideal parallax becomes zero.

FIG. 24 is an overall schematic flowchart of the calibration methodaccording to the second embodiment. The stereo camera 10 acquires thecaptured image (Step S201). Specifically, the first camera 1 acquiresthe first captured image, and the second camera 2 acquires the secondcaptured image.

Next, the calibration device 30 measures the relative position betweenthe stereo camera 10 and the object (Step S202). Specifically, thecalibration device 30 measures the relative position between the opticalcenter O₀ of the first camera 1 of the stereo camera 10 and themeasurement tool 120 at Step S32 to Step S34 that are described later.

Next, the calibration device 30 determines a calibration parameter basedon the relative position (Step S203). Specifically, the calibrationdevice 30 determines a calibration parameter for correcting at least oneof the first captured image and the second captured image such that theparallax that is calculated by detecting a corresponding point that isin the second captured image and that corresponds to a point in thefirst captured image by using a shading pattern of the measurement tool120 (the first member 102), or the like, matches the ideal parallax thatindicates the ideal parallax based on the relative position that ismeasured at Step S202.

Next, an explanation is given, with reference to the flowchart, of thedetails of the calibration method for the calibration device 30according to the second embodiment. FIG. 25 is a flowchart thatillustrates an example of the calibration method for the calibrationdevice 30 according to the second embodiment.

The receiving unit 131 receives the captured image (the first capturedimage and the second captured image) that includes the measurement tool120 as an object (Step S31). Furthermore, the receiving unit 131receives the distance information d_(c) (Step S32).

Next, the measuring unit 136 measures a deviation of the direction ofthe measurement tool 120 from the facing position of the stereo camera10 (Step S33). Specifically, the measuring unit 136 determines adirection (normal vector) that is perpendicular to the surface of themeasurement tool 120 (the angle measurement plate 101) by using themethod that is illustrated in FIG. 21 on the basis of the position ofthe maximum brightness in the first captured image, thereby measuring adeviation of the direction of the measurement tool 120.

Next, the measuring unit 136 determines the equation of the plane thatindicates the position of the measurement tool 120 in the first-cameracoordinate system (the coordinate system in which the optical center O₀of the first camera 1 is an origin) by using the method that isillustrated in FIG. 22 on the basis of the normal vector and thedistance information d_(c) (Step S34).

Next, the ideal-parallax calculating unit 137 uses the method that isillustrated in FIG. 3 to calculate an ideal parallax that indicates theparallax in a case where an image of the plane represented by theequation that is determined at Step S34 is captured (Step S35). Next,the parallax calculating unit 138 uses a shading pattern of themeasurement tool 120 (the first member 102), or the like, to detect acorresponding point that is in the second captured image and thatcorresponds to a point in the first captured image, thereby calculatinga parallax (Step S36).

Next, the determining unit 139 determines a calibration parameter forcorrecting at least one of the first captured image and the secondcaptured image such that the difference between the parallax and theideal parallax becomes zero (Step S37).

The overall flow of the calibration method according to the secondembodiment that uses the above-described measurement tool 120 and theabove-described calibration device 30 is the same as that is explainedin FIG. 12 according to the first embodiment; therefore, its explanationis omitted.

As described above, with the calibration method according to the secondembodiment, a deviation of the direction of the measurement tool 120from the facing position of the stereo camera 10 is measured on thebasis of the position of the maximum brightness of the first capturedimage. Furthermore, a calibration parameter for calibrating the stereocamera 10 is determined on the basis of the parallax that is calculatedfrom the first captured image and the second captured image and theideal parallax in which a deviation of the direction of the measurementtool 120 is taken into account. Thus, it is possible to easily calculatea calibration parameter with high accuracy with respect to the stereocamera 10 that is installed in a vehicle for which it is difficult toensure the installation location accuracy. Furthermore, according to thepresent embodiment, it is possible to achieve a calibration with higheraccuracy by using a simple configuration with respect to the stereocamera 10 that is not installed in a vehicle, or the like. Furthermore,the hole of the angle measurement plate is a circular hole in thepresent embodiment; however, this is not a limitation. For example, asquare hole, or the like, may be used.

Third Embodiment

Next, a third embodiment is explained. In the third embodiment, anexplanation is given of a case where a measurement tool 220 (the anglemeasurement plate 101) is used, which has a different configuration thanthat of the measurement tool 120 (the angle measurement plate 101)according to the second embodiment. In the explanation of the thirdembodiment, the point that is different from that of the secondembodiment is explained.

FIG. 26 is a cross-sectional view that illustrates a cross-sectionalsurface of an angle measurement plate 201 according to the thirdembodiment. As the front view of the angle measurement plate 201 is thesame as that in FIG. 15B, it is omitted. The angle measurement plate 201includes the light source 103, a light shielding plate 202, atransparent glass 203, and a light shielding plate 204. As the lightsource 103 is the same as that in the second embodiment, its explanationis omitted. The light shielding plate 202, the transparent glass 203,and the light shielding plate 204 correspond to the second member 104according to the second embodiment.

The angle measurement plate 201 according to the third embodiment uses atransparent glass plate that has opaque light shielding areas (the lightshielding plate 202 and the light shielding plate 204) arranged on bothsurfaces thereof. The transparent glass 203 is provided to fix thepositions of the light shielding plate 202 and the light shielding plate204. The gap between the light shielding plate 202 and the lightshielding plate 204 is filled with glass; thus, it is possible to haveresistance to mechanical displacements and to reduce measurement errorsthat are caused by a temperature, deformation with time, or the like.The transparent glass 203 may be any transparent object.

The angle measurement plate 201 according to the third embodiment is thesame as the angle measurement plate 101 according to the firstembodiment in that, because of the holes of the light shielding surfaceson the front and back sides, light that enters in a directionsubstantially perpendicular to the angle measurement plate 201 is passedand transmitted through the opposing hole (e.g., a hole 208 and a hole209). However, as light is refracted at the boundary surface between thetransparent glass 203 and air, the brightness distribution of thecaptured image is different from that of the second embodiment.

FIG. 27 is a diagram that illustrates the angle at which light isrefracted. As already known (Snell's law), the direction in which thetransmitted light is output is changed due to the refraction on theglass-air boundary surface. If the specific refractive index is R, therelationship between angles θ1 and θ2 of FIG. 27 is the followingEquation (11).

$\begin{matrix}{\frac{\sin \left( \theta_{1} \right)}{\sin \left( \theta_{2} \right)} = R} & (11)\end{matrix}$

Therefore, if the overall thickness of the light shielding plate 202,the transparent glass 203, and the light shielding plate 204 is the sameas that of the second member 104 according to the second embodiment, andso is the diameter of the holes, farther holes can be seen due to therefraction of glass (see FIG. 28). Therefore, the size of the foot ofthe mountain of the brightness distribution of the captured image islarger than that in the second embodiment. However, the samecharacteristics are obtained such that the position of the brightnesspeak corresponds to the normal direction of the light shielding plate204 regardless of the position of the optical center O₀ of the stereocamera 10 (the first camera 1).

Unlike the second member 104 according to the first embodiment, theangle measurement plate 201 according to the third embodiment permits atransmission of light that passes through the hole that does not face itand is located in a direction that is indicated by a dotted line of FIG.28 other than a front direction. For example, with regard to a hole 207,light through the hole 209 that is opposed to the hole 208 istransmitted through the hole 207 due to the effect of refraction.

Therefore, unlike the case of the second embodiment, a moire patternwhere light and dark are periodically repeated is observed from thecaptured image of the angle measurement plate 201. Therefore, there is apossibility that multiple brightness peaks are present within the imagecapturing area. However, if the accuracy of the installation angles ofthe stereo camera 10 (the first camera 1) and the measurement tool 220is previously determined, it is possible to prevent the position of thebrightness peak of the facing hole from being mistaken for thebrightness peak that corresponds to an adjacent hole by considering theperiod of moire that corresponds to the moving range of the position ofthe brightness peak that is based on the range of an installationdeviation.

FIG. 29 is a diagram that illustrates the relationship between a tilt ofthe measurement tool 220 and a deviation of the position of thebrightness peak. As in FIG. 29, if the installation angle of themeasurement tool 220 has an angle deviation of equal to or less than ±Xdegrees from the facing direction, the position of the brightness peakis present in a range of equal to or less than X degrees from the facingposition (the center of the screen). Specifically, as in FIG. 30, theinterval of holes may be adjusted such that the brightness peak of anadjacent hole is located at a position more than twice as far away asfrom the expected positional deviation of the brightness peak. Theposition of the brightness peak of an adjacent hole is determined byusing an angle ϕ of FIG. 31 (specifically, f tan (ϕ)), and the angle ϕhas the relationship of the following Equation (12) with a lightshielding surface's hole pitch (p), a glass plate thickness (d), and aglass refractive index (n).

$\begin{matrix}{\frac{\sin (\Phi)}{\sin \left( {\arctan \left( {p/d} \right)} \right)} = n} & (12)\end{matrix}$

According to Equation (12), it is understood that the glass platethickness (d) and the hole pitch (p) may be determined such that thefollowing Equation (13) is satisfied.

Φ=arcsin(n sin(arctan(p/d)))>2X  (13)

In the above-described explanation, only the accuracy of theinstallation angle is considered to expect the position of the adjacentbrightness peak. However, in actuality, it is necessary to expect therange in which the brightness peak indicating the facing position ispresent by considering every conceivable installation deviations, suchas the installation angle of the stereo camera 10, a translationaldeviation of the measurement tool 220, or a translational deviation ofthe stereo camera 10 other than the installation angle of themeasurement tool 220. Furthermore, the glass plate thickness (d) and thehole pitch (p) are determined such that the brightness peak of anadjacent hole does not fall within the range of the expected brightnesspeak position; thus, it is possible to uniquely determine the positionof the brightness peak within the expected range that corresponds to thehole facing the stereo camera 10.

It is possible to use a technology, such as printing or photo-etching,as a method for forming a light shielding area on a flat plate when thelight shielding plate 202 (204) is formed. With these technologies, ingeneral, it is easy to form a hole with a small diameter or a narrowpitch, compared to the second member 104 according to the secondembodiment that is formed by using a method of forming a hole in a thickplate with a drill, or the like. The size of the mountain of brightnessis determined according to the ratio of the hole diameter to the platethickness (and the refractive index). For example, if the platethickness (the overall thickness of the light shielding plate 202, thetransparent glass 203, and the light shielding plate 204) is 6 mm andthe hole radius is 0.05 mm, substantially the same brightnessdistribution as that in the case of the second embodiment can beobtained.

As explained above, with regard to the measurement tool 220 according tothe third embodiment, even if the angle measurement plate 201 is a morelightweight, smaller, and thinner plate compared to the anglemeasurement plate 101 according to the second embodiment, it is possibleto achieve the equivalent calibration accuracy to that of the anglemeasurement plate 101 according to the second embodiment.

Furthermore, to simply block light, the light shielding plate 202 andthe light shielding plate 204 may be installed at the same positions asthe light shielding areas on both sides of glass without installing thetransparent glass 203.

Finally, an explanation is given of an example of a hardwareconfiguration of the calibration device 30 according to the first to thethird embodiments. FIG. 32 is a diagram that illustrates an example ofthe hardware configuration of the stereo camera 10 and the calibrationdevice 30 according to the first to the third embodiments.

The calibration device 30 according to the first to the thirdembodiments includes a control device 51, a primary storage device 52,an auxiliary storage device 53, a display device 54, an input device 55,and a communication device 56. The control device 51, the primarystorage device 52, the auxiliary storage device 53, the display device54, the input device 55, and the communication device 56 are connectedto one another via a bus 57.

The stereo camera 10 according to the first to the third embodimentsincludes an image capturing device 151, an image capturing device 152, acommunication device 153, a control device 154, a primary storage device155, an auxiliary storage device 156, a display device 157, and an inputdevice 158. The image capturing device 151, the image capturing device152, the communication device 153, the control device 154, the primarystorage device 155, the auxiliary storage device 156, the display device157, and the input device 158 are connected to one another via a bus159.

The control device 51 (the control device 154) is a CPU. The controldevice 51 (the control device 154) executes a program that is read fromthe auxiliary storage device 53 (the auxiliary storage device 156) intothe primary storage device 52 (the primary storage device 155). Theprimary storage device 52 (the primary storage device 155) is a memory,such as a ROM or RAM. The auxiliary storage device 53 (the auxiliarystorage device 156) is an HDD (Hard Disk Drive), a memory card, or thelike. The display device 54 (the display device 157) displays the stateof the calibration device 30 (the stereo camera 10), or the like. Theinput device 55 (the input device 158) receives an input from a user.The communication device 56 of the calibration device 30 and thecommunication device 153 of the stereo camera 10 communicate with eachother via a wired or wireless network.

The image capturing device 151 corresponds to the first camera 1 (seeFIG. 2). The image capturing device 152 corresponds to the second camera2 (see FIG. 2).

A program that is to be executed by the stereo camera 10 and thecalibration device 30 according to the first to the third embodiments isprovided as a computer program product by being stored, in the form of afile that is installable and executable, in a recording medium readableby a computer, such as a CD-ROM, a memory card, a CD-R, or a DVD(Digital Versatile Disk).

Furthermore, a configuration may be such that a program that is to beexecuted by the stereo camera 10 and the calibration device 30 accordingto the first to the third embodiments is stored in a computer connectedvia a network such as the Internet and is provided by being downloadedvia the network. Moreover, a configuration may be such that a programthat is to be executed by the stereo camera 10 and the calibrationdevice 30 according to the first to the third embodiments is providedvia a network such as the Internet without being downloaded.

A configuration may be such that a program for the stereo camera 10 andthe calibration device 30 according to the first to the thirdembodiments is provided such that it is installed in a ROM, or the like,in advance.

A program that is to be executed by the calibration device 30 accordingto the first embodiment has a modular configuration that includes theabove-described functional blocks (the receiving unit 31, thefirst-camera position calculating unit 32, the first-camera directioncalculating unit 33, the second-camera position calculating unit 34, thesecond-camera direction calculating unit 35, the ideal-parallaxcalculating unit 37, the parallax calculating unit 38, and thedetermining unit 39). Furthermore, a program that is to be executed bythe calibration device 30 according to the second and the thirdembodiments has a modular configuration that includes theabove-described functional blocks (the receiving unit 131, the measuringunit 136, the ideal-parallax calculating unit 137, the parallaxcalculating unit 138, and the determining unit 139).

Furthermore, a program that is to be executed by the stereo camera 10according to the first to the third embodiments has a modularconfiguration that includes the above-described functional blocks (thecorrecting unit 5 and the calculating unit 6).

With respect to the above-described functional blocks, in terms ofactual hardware, the control device 51 (the control device 154) readsthe program from the above-described recording medium and executes it soas to load the above-described functional blocks into the primarystorage device 52 (the primary storage device 155). That is, theabove-described functional blocks are generated in the primary storagedevice 52 (the primary storage device 155).

Furthermore, all or some of the above-described functional blocks may beimplemented by hardware, such as an IC (Integrated Circuit), withoutbeing implemented by software.

An embodiment provides an advantage that a high-accuracy calibrationparameter for calibrating a stereo camera can be calculated.

Although the invention has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

REFERENCE SIGNS LIST

-   -   1 FIRST CAMERA    -   2 SECOND CAMERA    -   3 STORAGE UNIT    -   4 EXTERNAL I/F    -   5 CORRECTING UNIT    -   6 CALCULATING UNIT    -   10 STEREO CAMERA    -   20 MEASUREMENT TOOL    -   21 MARK    -   22 DISTANCE MEASUREMENT DEVICE    -   23 MEASUREMENT POINT    -   24 LASER BEAM    -   30 CALIBRATION DEVICE    -   31 RECEIVING UNIT    -   32 FIRST-CAMERA POSITION CALCULATING UNIT    -   33 FIRST-CAMERA DIRECTION CALCULATING UNIT    -   34 SECOND-CAMERA POSITION CALCULATING UNIT    -   35 SECOND-CAMERA DIRECTION CALCULATING UNIT    -   36 DISTANCE CALCULATING UNIT    -   37 IDEAL-PARALLAX CALCULATING UNIT    -   38 PARALLAX CALCULATING UNIT    -   39 DETERMINING UNIT    -   51 CONTROL DEVICE    -   52 PRIMARY STORAGE DEVICE    -   53 AUXILIARY STORAGE DEVICE    -   54 DISPLAY DEVICE    -   55 INPUT DEVICE    -   56 COMMUNICATION DEVICE    -   57 BUS    -   101 ANGLE MEASUREMENT PLATE    -   102 FIRST MEMBER (CHART)    -   103 LIGHT SOURCE    -   104 SECOND MEMBER (LIGHT SHIELDING PLATE)    -   120 MEASUREMENT TOOL    -   131 RECEIVING UNIT    -   136 MEASURING UNIT    -   137 IDEAL-PARALLAX CALCULATING UNIT    -   138 PARALLAX CALCULATING UNIT    -   139 DETERMINING UNIT    -   201 ANGLE MEASUREMENT PLATE    -   202 LIGHT SHIELDING PLATE    -   203 TRANSPARENT GLASS    -   204 LIGHT SHIELDING PLATE    -   220 MEASUREMENT TOOL

CITATION LIST Patent Literature [Patent Literature 1] Japanese PatentNo. 4109077

1. A calibration method for calibrating a stereo camera, the calibrationmethod comprising: capturing, using the stereo camera, an imageincluding an object placed so as to fall within an image capturing areaof the stereo camera; acquiring a first distance from the object to thestereo camera using a second distance from the object to an intermediatemeasurement point that is located between the object and the stereocamera; and determining a calibration parameter for calibrating thestereo camera based on the first distance and the captured image.
 2. Thecalibration method according to claim 1, wherein the second distancefrom the object to the intermediate measurement point is measured usinglaser light.
 3. The calibration method according to claim 1, wherein thesecond distance from the object to the intermediate measurement point ismeasured using a measuring tape.
 4. The calibration method according toclaim 1, further comprising measuring a third distance from theintermediate measurement point to the stereo camera.
 5. The calibrationmethod according to claim 1, wherein the third distance from theintermediate measurement point to the stereo camera is measured inadvance.
 6. The calibration method according to claim 1, wherein thestereo camera is mounted inside a movable object, and the object islocated outside the movable object.
 7. The calibration method accordingto claim 6, wherein the stereo camera is installed inside a frontwindshield of a vehicle and the intermediate measurement point isprovided on the front windshield of the vehicle.
 8. A calibration devicefor calibrating a stereo camera, the calibration device comprising: animage capturer configured to capture, using the stereo camera, an imageincluding an object placed so as to fall within an image capturing areaof the stereo camera; and processing circuitry configured to acquire afirst distance from the object to the stereo camera using a seconddistance from the object to an intermediate measurement point that islocated between the object and the stereo camera, and determine acalibration parameter for calibrating the stereo camera based on thefirst distance and the captured image.
 9. A non-transitorycomputer-readable medium storing a program that, when executed byprocessing circuitry, causes the processing circuitry to calibrate astereo camera by executing a method comprising: capturing, with thestereo camera, an image including an object placed so as to fall withinan image capturing area of the stereo camera; acquiring a first distancefrom the object to the stereo camera using a second distance from theobject to an intermediate measurement point that is located between theobject and the stereo camera; and determining a calibration parameterfor calibrating the stereo camera based on the first distance and thecaptured image.